Papers with spoken language understanding
Copied to clipboard
| Challenge: | Spoken Language Understanding (SLU) is a task-oriented dialogue system . open-source toolkit provides a unified, modularized, and extensible toolkit for SLU . |
| Approach: | They introduce an open-source toolkit to provide a unified toolkit for spoken language understanding. |
| Outcome: | The proposed toolkit unifies 10 models for both single-intent and multi-intention scenarios. |
Copied to clipboard
| Challenge: | lexicons or gazettes are used to improve slot tagging in spoken language understanding systems. |
| Approach: | They develop models that encode lexicon information as neural features for use in a long-short term memory neural network. |
| Outcome: | The proposed models improve slot tagging with lexicons and gazettes . the results could be used to improve other natural language applications . |
Copied to clipboard
| Challenge: | Existing methods to integrate hypotheses into speech recognition systems are noisy and can cause information loss. |
| Approach: | They propose to integrate hypotheses into multi-task learning and transfer learning to improve performance. |
| Outcome: | The proposed model improves domain and intent classification by 19% and 37% compared to current methods . the proposed model could recover transcription and rewrite the query for a better understanding . |
Copied to clipboard
| Challenge: | Using a pointer-generator network, we model the reference resolution task as a dialogue context-aware user query reformulation task. |
| Approach: | They propose a pointer-generator network and a novel multi-task learning setup to model dialogue state tracking and referring expression resolution tasks using a dialogue context-aware user query reformulation task. |
| Outcome: | The proposed model improves absolute F1 on internal and public benchmarks. |
Copied to clipboard
| Challenge: | a new meaning representation language for spoken language is introduced for Alexa . AMRL provides a common representation for how people communicate in spoken language . there is no mechanism to represent ambiguity, forcing the choice of a fixed interpretation for ambiguous utterances. |
| Approach: | They introduce a meaning representation for spoken language, the Alexa meaning represent language . they use a spoken language dataset to collect a sample of utterances from eight domains . |
| Outcome: | The proposed representation provides a common representation for spoken language understanding . it supports cross-domain queries, fine-grained types, complex utterances and composition . the proposed representation was released to developers at a trade show in 2016 . |
Copied to clipboard
| Challenge: | Larger encoders can improve accuracy for spoken language understanding (SLU) but are difficult to use given the inference latency constraints of online systems. |
| Approach: | They propose to use a larger 170M parameter BERT encoder that shares representations across languages, domains and tasks for SLU. |
| Outcome: | The proposed encoders achieve state-of-the-art performance on numerous NLP tasks. |
Copied to clipboard
| Challenge: | Slot filling is a key subtask in spoken language understanding (SLU) . recent advent of speech-based large language models has opened new avenues for speech understanding . |
| Approach: | They propose to improve slot-filling task by creating an empirical upper bound for the task . they propose to use a speech-based large language model to integrate speech and text modalities . |
| Outcome: | The proposed model improves slot filling performance while reducing generalization gaps. |
Copied to clipboard
| Challenge: | Multilingual automatic speech recognition (ASR) in the medical domain is a critical foundational task, serving a wide range of downstream applications such as speech translation, spoken language understanding, and voice-activated assistants. |
| Approach: | They present the first multilingual medical ASR dataset and the first collection of small-to-large end-to end medical APR models spanning five languages: Vietnamese, English, German, French, and Mandarin Chinese. |
| Outcome: | The proposed model covers Vietnamese, English, German, French, and Mandarin Chinese, and is the first multilingual ASR dataset across five languages. |
Copied to clipboard
| Challenge: | a new study addresses the challenge of learning semantic representations from speech signals . speech-based semantic representation can be used for speech mining and spoken language understanding . |
| Approach: | They propose a multimodal sequential autoencoder that converts speech signals into hidden units . they propose s-HuBERT to induce meaning through knowledge distillation . |
| Outcome: | The proposed model achieves a moderate correlation with human judgments without labels or transcriptions. |
Copied to clipboard
| Challenge: | Existing approaches to slot filling training on large scale data are inefficient and require multiple trainings. |
| Approach: | They propose a slot filling model that transfers previously learned knowledge to a small size expanded component and enables it to be fast trained to learn from new data. |
| Outcome: | The proposed model outperforms existing models on two benchmark datasets by 4.24% and 3.03% on the same dataset. |
Copied to clipboard
| Challenge: | a dialogue state tracker is a core component in most of today's spoken dialogue systems . slot-filling dialogues are composed of a predefined set of slots that need to be filled through the conversation . |
| Approach: | They propose an E2E architecture that extracts unknown slot values while still achieving state-of-the-art accuracy on the standard DSTC2 benchmark. |
| Outcome: | The proposed architecture achieves state-of-the-art accuracy on the DSTC2 benchmark while retaining predefined slot values. |
Copied to clipboard
| Challenge: | End-to-end (E2E) spoken language understanding models are constrained by the cost of collecting speech-semantics pairs. |
| Approach: | They propose a model that learns E2E SLU without speech-semantics pairs . they propose cross-modal selective self-training (CMSST) to address imbalance and noise issues . |
| Outcome: | The proposed model learns E2E SLU without speech-semantics pairs . the proposed model requires the domains of speech-text and text-sensitization to match . |
Copied to clipboard
| Challenge: | Covariate drift can occur when there is a drift between training and testing regarding what users request or how they request it. |
| Approach: | They propose a method that exploits natural variations in data to create a covariate drift in spoken language understanding datasets. |
| Outcome: | The proposed method improves robustness against covariate drift in spoken language understanding (SLU) it shows that a state-of-the-art model suffers performance loss under this drift. |
Copied to clipboard
| Challenge: | Recent studies leverage large language models with multi-tasking capabilities, using natural language prompts to guide the model’s behavior and surpassing performance of task-specific models. |
| Approach: | They adapt a pre-trained automatic speech recognition model to additional tasks using single-token task specifiers. |
| Outcome: | The proposed model can generalize to new datasets and languages for seen task types. |
Copied to clipboard
| Challenge: | Slot filling and intent detection are two main tasks in spoken language understanding systems. |
| Approach: | They propose a non-autoregressive slot filling model with two-pass iteration mechanism to handle uncoordinated slots problem. |
| Outcome: | The proposed model significantly outperforms previous models in slot filling task while speeding up decoding. |
Copied to clipboard
| Challenge: | Existing models focus on the single intent scenario, ignoring the fine-grained multiple intents information integration for token-level slot prediction. |
| Approach: | They propose an Adaptive Graph-Interactive Framework for joint multiple intent detection and slot filling . they propose an intent-slot graph interaction layer to model the strong correlation between the slot and intents . |
| Outcome: | The proposed framework improves on three multi-intent datasets and new state-of-the-art performance on single-intention datasets. |
Copied to clipboard
| Challenge: | Existing approaches to prevent catastrophic forgetting in neural networks are based on the stability-plasticity dilemma, but only a limited size of old data is available. |
| Approach: | They propose a Continual Learning Long Short Term Memory cell in Recurrent Neural Network (RNN) that considers the state of each individual task's output gates and the correlation of the states between tasks. |
| Outcome: | The proposed method significantly improves on spoken language understanding tasks over state-of-the-art approaches. |
Copied to clipboard
| Challenge: | a growing number of studies address the spoken language understanding domain through a simple task like speech intent detection. |
| Approach: | They focus on the french MEDIA SLU dataset, which is distributed since 2005 . they propose a recipe for its use, including data preparation, training and evaluation scripts . |
| Outcome: | The MEDIA SLU dataset is used as a benchmark dataset for a large number of research projects. |
Copied to clipboard
| Challenge: | Existing methods for intent classification are expensive to collect and train . evaluators have shown that the ability to detect out-of-domain utterances is limited . |
| Approach: | They propose to train a model with only IND data while supporting both intent classification and OOD detection. |
| Outcome: | The proposed model improves on existing models and strong baselines on four datasets. |
Copied to clipboard
| Challenge: | Experimental results show that the combination of regular expressions and NNs improves learning effectiveness when a small number of training examples are available. |
| Approach: | They propose to combine a neural network (NN) with regular expressions (RE) to improve supervised learning for NLP by exploiting the rich expressiveness of REs at different levels within a NN. |
| Outcome: | The proposed approach significantly improves learning effectiveness when a small number of training examples are available. |
Copied to clipboard
| Challenge: | Intent detection and slot filling are two main tasks for building a spoken language understanding system. |
| Approach: | They propose a framework to incorporate intent information into slot filling tasks . they use a joint model with Stack-Propagation to capture intent semantic knowledge . |
| Outcome: | The proposed model outperforms existing models on two publicly available datasets and outperformed existing models by a large margin. |
Copied to clipboard
| Challenge: | Recent advances in speech processing require extensive offline training . however, these models struggle to retain their previously acquired knowledge when learning new tasks continuously. |
| Approach: | They propose a method that relies on experience replay and contrastive learning to preserve the learned representations by pulling closer samples from the same class and pushing away the others. |
| Outcome: | The proposed method preserves the learned representations by pulling closer samples from the same class and pushing away the others. |
Copied to clipboard
| Challenge: | Existing approaches to filling slots that take on values from a virtually unlimited set have been lacking in the natural language area. |
| Approach: | They propose a new attention-based recurrent neural network (RNN) model that captures the concept: Understanding the role of a word may vary according to how long a reader focuses on a particular part of . sentence. |
| Outcome: | The proposed model outperforms existing models with respect to discovering ‘open-vocabulary’ slots without any external information, such as a named entity database or knowledge base. |
Copied to clipboard
| Challenge: | Existing work on slot filling and intent detection builds joint models without prior knowledge of linguistic knowledge. |
| Approach: | They propose a joint model that integrates syntactic structure for learning slot filling and intent detection jointly. |
| Outcome: | The proposed model outperforms existing models on two public benchmark datasets and further improves on slot filling and intent detection. |
Copied to clipboard
| Challenge: | a study aims to develop a language transferring system to avoid the trouble of acquiring and labeling a new big SLU corpus . general-purpose translators cannot handle the lot of semantic labels, not to mention cultural differences . a RL-based language transfer method can be used to adapt the adapted translator to a target language . |
| Approach: | They propose to use reinforcement learning to adapt a spoken language understanding model to a target language. |
| Outcome: | The proposed language transferring method improves domain classification accuracy by 22% compared with naive translation . the proposed language transfer method can be used on Chinese to English translators with more proper slot tags . |
Copied to clipboard
| Challenge: | Existing methods to improve robustness of models focus on a single dataset . but, there are few studies on how to combine merits of different datasets . |
| Approach: | They propose a federated learning framework that could unify datasets and tasks . they propose MV-Encoder as backbone of the framework to provide multi-granularity text representations . |
| Outcome: | The proposed framework improves on two SLU benchmark datasets and federated learning settings. |
Copied to clipboard
| Challenge: | Latent Synthesis is an efficient textual data utilization framework for end-to-end speech processing models . labeled speech data are scarcer and more expensive for collection compared to textual ones . |
| Approach: | They propose a textual data utilization framework for E2E speech processing models . they train a latent synthesizer to convert textual information into an intermediate latent representation . |
| Outcome: | The proposed framework improves on low-resource speech recognition and spoken language understanding tasks. |
Copied to clipboard
| Challenge: | Existing models that use contextual information of dialogues to improve spoken language understanding (SLU) select the wrong history when the histories are similar in content. |
| Approach: | They propose time-aware models that automatically learn the latent time-decay function of the history without a manual time- decay. |
| Outcome: | The proposed models achieve higher F1 scores than state-of-the-art models on a benchmark dataset . |
Copied to clipboard
| Challenge: | Existing models rely on annotated training data, limiting their scalability to low-resource languages. |
| Approach: | They propose a method termed SoGo for zero-shot cross-lingual SLU that uses keywords as substitution options to extract keywords and a token-level alignment strategy to ensure grammatical coherence. |
| Outcome: | The proposed method improves zero-shot cross-lingual SLU across nine languages on MultiATIS++. |
Copied to clipboard
| Challenge: | Existing methods to simulate automatic speech recognition errors from manual transcriptions are not available during training of the SLU model. |
| Approach: | They propose to use acoustic and linguistic word embeddings to define a similarity measure between words to predict ASR confusions. |
| Outcome: | The proposed method significantly improves the performance of spoken language understanding systems. |
Copied to clipboard
| Challenge: | Existing models for SLU use explicit memory representations, but the context memory is under-exploited. |
| Approach: | They propose a dialogue logistic inference task to consolidate the context memory with SLU in a multi-task framework. |
| Outcome: | The proposed model improves slot filling and domain classification performance in a multi-task framework. |
Copied to clipboard
| Challenge: | Existing models for slot filling and intent detection lack bi-directional interrelated connections between the intent and slots. |
| Approach: | They propose a bi-directional interrelated model for slot filling and intent detection that uses an SF-ID network to establish direct connections between the two tasks to promote each other mutually. |
| Outcome: | The proposed model improves on ATIS and Snips datasets in sentence-level semantic frame accuracy and improves performance on the two tasks. |
Copied to clipboard
| Challenge: | Recent work on spoken video grounding challenges extracting semantic information from speech . previous studies focused on textual queries, but recent work focuses on spoken queries . |
| Approach: | They propose a framework for weakly-supervised spoken video grounding to represent cross-modal semantics without expensive temporal annotations. |
| Outcome: | The proposed framework is more efficient than existing methods. |
Copied to clipboard
| Challenge: | Existing approaches to multiple intent detection and slot filling focus on task-specific components to capture the relationships between intents and slots. |
| Approach: | They propose a Unified Generative framework that captures the relationships between intents and slots in an utterance and formulates the task as a question-answering problem. |
| Outcome: | The proposed framework surpasses baselines on full-data and multi-intent benchmarks on 5-shot and 10-shot scenarios. |
Copied to clipboard
| Challenge: | Existing approaches to translate spoken language understanding into low-resource languages are limited to implicit alignment and disregard the inherent semantic structure in SLU. |
| Approach: | They propose to model utterance-slot-word structure by a multi-level contrastive learning framework . they also propose a label-aware joint model leveraging label semantics to enhance alignment . |
| Outcome: | The proposed model improves performance on two zero-shot cross-lingual datasets. |
Copied to clipboard
| Challenge: | Spoken language understanding evaluation (SLUE) benchmarks are used to benchmark complex spoken language understanding tasks on natural speech. |
| Approach: | They propose a set of benchmark tasks to evaluate spoken language understanding on natural speech . they use pre-trained speech foundation models to evaluate the utility of different SFMs . |
| Outcome: | The proposed framework outperforms pre-trained speech foundation models on natural speech . the proposed framework also outperformed self-supervised SFMs on the sequence generation tasks . |
Copied to clipboard
| Challenge: | Spoken language understanding (SLU) is a crucial task in task-oriented dialogue systems. |
| Approach: | They propose an ASR-Robust SLU framework based on the mixture-of-experts technique to generate additional transcripts from clean transcripts and use it to weigh the representations of the generated transcripts, ASR transcripts . |
| Outcome: | The proposed framework achieves state-of-the-art on three benchmark SLU datasets. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities in language generation and general task performance, but their application to spoken language understanding remains challenging. |
| Approach: | They propose an Entity-level Language Model framework which reformulates slot-filling as an entity recognition task and introduces a new concept, Chain of Intent, to enable step-by-step multi-intent recognition. |
| Outcome: | The proposed framework outperforms strong baselines such as Uni-MIS and achieves gains of 3.7% and 3.1% on MixATIS and MixSNIPS. |
Copied to clipboard
| Challenge: | Existing approaches to slot filling are limited due to data scarcity and timeconsuming efforts. |
| Approach: | They propose a framework that harnesses the power of a small model to augment inferential capabilities of LLMs without additional training. |
| Outcome: | The proposed framework improves slot filling performance on a spoken language dataset and a NER dataset. |
Copied to clipboard
| Challenge: | Existing methods to increase the robustness of pre-trained language models (PLMs) against unseen ASR systems produce noisy inputs for SLU models, which can significantly degrade their performance. |
| Approach: | They propose to introduce ASR-plausible noises into pre-trained language models by cutting off the non-causal effect of noises. |
| Outcome: | The proposed method improves the robustness and generalizability of SLU models against unseen ASR systems by cutting off the non-causal effect of noises. |
Copied to clipboard
| Challenge: | Speech Large Language Models (SpeechLLMs) have emerged as dominant speech processing approaches. |
| Approach: | They compare self-supervised learning-based discrete and continuous features . they compare performance across six spoken language understanding-related tasks . |
| Outcome: | The proposed models outperform discrete tokens and continuous features in six spoken language understanding-related tasks. |
Copied to clipboard
| Challenge: | Recent studies show voice assistants do not perform equally well for everyone . however, research on demographic robustness of speech technologies is still scarce . |
| Approach: | They propose a statistical method to detect demographic bias using a large dataset with controlled demographic tags. |
| Outcome: | The proposed method shows statistically significant differences in performance across age, dialectal region and ethnicity. |
Copied to clipboard
| Challenge: | a growing number of recorded human speech is recorded for automated processing, resulting in errors in the transcripts . a configurable framework is proposed to analyze transcript noise impact across noise levels and transcript-cleaning techniques. |
| Approach: | They propose a configurable framework for assessing task models in diverse noisy settings . framework facilitates investigation of task model behavior, which can support effective SLU solutions. |
| Outcome: | The proposed framework can analyze model behavior in various noise levels and transcript-cleaning techniques. |
Copied to clipboard
| Challenge: | Pretrained acoustic models are increasingly used for downstream speech tasks such as automatic speech recognition, speech translation, spoken language understanding or speech parsing. |
| Approach: | They propose to probing a pretrained acoustic model for French for syntactic information using the Orféo treebank. |
| Outcome: | The proposed model is trained on 7k hours of spoken French and obtained reasonable results on tasks that require higher level linguistic knowledge. |